Forecasting daily COVID-19 cases with gradient boosted regression trees and other methods: evidence from U.S. cities.

Introduction: There is a vast literature on the performance of different short-term forecasting models for country specific COVID-19 cases, but much less research with respect to city level cases. This paper employs daily case counts for 25 Metropolitan Statistical Areas (MSAs) in the U.S. to evaluate the efficacy of a variety of statistical forecasting models with respect to 7 and 28-day ahead predictions. Methods: This study employed Gradient Boosted Regression Trees (GBRT), Linear Mixed Effects (LME), Susceptible, Infectious, or Recovered (SIR), and Seasonal Autoregressive Integrated Moving Average (SARIMA) models to generate daily forecasts of COVID-19 cases from November 2020 to March 2021. Results: Consistent with other research that have employed Machine Learning (ML) based methods, we find that Median Absolute Percentage Error (MAPE) values for both 7-day ahead and 28-day ahead predictions from GBRTs are lower than corresponding values from SIR, Linear Mixed Effects (LME), and Seasonal Autoregressive Integrated Moving Average (SARIMA) specifications for the majority of MSAs during November-December 2020 and January 2021. GBRT and SARIMA models do not offer high-quality predictions for February 2021. However, SARIMA generated MAPE values for 28-day ahead predictions are slightly lower than corresponding GBRT estimates for March 2021. Discussion: The results of this research demonstrate that basic ML models can lead to relatively accurate forecasts at the local level, which is important for resource allocation decisions and epidemiological surveillance by policymakers.

Estimating the Effects of Non-Pharmaceutical Interventions and Population Mobility on Daily COVID-19 Cases: Evidence from Ontario.

This study uses coronavirus disease 2019 (COVID-19) case counts and Google mobility data for 12 of Ontario's largest Public Health Units from Spring 2020 until the end of January 2021 to evaluate the effects of non-pharmaceutical interventions (NPIs; policy restrictions on business operations and social gatherings) and population mobility on daily cases. Instrumental variables (IV) estimation is used to account for potential simultaneity bias, because both daily COVID-19 cases and NPIs are dependent on lagged case numbers. IV estimates based on differences in lag lengths to infer causal estimates imply that the implementation of stricter NPIs and indoor mask mandates are associated with reductions in COVID-19 cases. Moreover, estimates based on Google mobility data suggest that increases in workplace attendance are correlated with higher case counts. Finally, from October 2020 to January 2021, daily Ontario forecasts from Box-Jenkins time-series models are more accurate than official forecasts and forecasts from a susceptible-infected-removed epidemiology model.

Cette étude cherche à évaluer les effets des interventions non pharmaceutiques (INPs; restrictions sur les activités commerciales et rassemblements sociaux) et de la mobilité de la population sur le nombre de cas d'infection par jour, en utilisant les nombres de cas d'infection par la maladie à coronavirus 2019 (COVID-19) et les données de mobilité de Google pour 12 des plus grands Bureaux de Santé publique de l'Ontario entre le printemps 2020 et la fin janvier 2021. La méthode des variables instrumentales (VI) permet de rendre compte d'un biais potentiel de simultanéité puisque les taux quotidiens de COVID-19 et les INPs dépendent, tous les deux, du nombre de cas décalés. Les estimations par les VI basées sur les différences de durée des décalages d'ajustement pour inférer des estimations causales impliquent que de plus strictes INPs et le port obligatoire du masque dans les endroits fermés sont associés à une réduction de cas d'infection. Par ailleurs, Les estimations basées sur les données de mobilité de Google montrent que la présence accrue sur le lieu du travail est corrélée avec un plus grand nombre de cas d'infection. Finalement, d'octobre 2020 à Janvier 2021, les prévisions faites à partir de modèles de Box-Jenkins en série chronologique s'avèrent plus précises que les prévisions officielles et que celles utilisant le modèle épidémiologique susceptible ­ infecté ­ retiré.

Comparing Kaplan-Meier curves with the probability of agreement.

The probability of agreement has been used as an effective strategy for quantifying the similarity between the reliability of two populations. By contrast to hypothesis testing approaches based on P-values, the probability of agreement provides a more realistic assessment of similarity by emphasizing practically important differences. In this article, we propose the use of the probability of agreement to evaluate the similarity of two Kaplan-Meier curves, which estimate the survival functions in two populations. This article extends the probability of agreement paradigm to right censored data and explores three different methods of quantifying uncertainty in the probability of agreement estimate. The first approach provides a convenient assessment based on large-sample normal-theory (LSNT), while the other two approaches are nonparametric alternatives based on ordinary and fractional random-weight bootstrap (FRWB) techniques. All methods are illustrated with examples for which comparing the survival curves of related populations is of interest and the efficacy of the methods are also evaluated through simulation studies. Based on these simulations we recommend point estimation using the proposed LSNT calculation and confidence interval estimation via the FRWB approach. We also provide a Shiny app that facilitates an automated implementation of the methodology.

Using multiple agreement methods for continuous repeated measures data: a tutorial for practitioners.

BACKGROUND: Studies of agreement examine the distance between readings made by different devices or observers measuring the same quantity. If the values generated by each device are close together most of the time then we conclude that the devices agree. Several different agreement methods have been described in the literature, in the linear mixed modelling framework, for use when there are time-matched repeated measurements within subjects. METHODS: We provide a tutorial to help guide practitioners when choosing among different methods of assessing agreement based on a linear mixed model assumption. We illustrate the use of five methods in a head-to-head comparison using real data from a study involving Chronic Obstructive Pulmonary Disease (COPD) patients and matched repeated respiratory rate observations. The methods used were the concordance correlation coefficient, limits of agreement, total deviation index, coverage probability, and coefficient of individual agreement. RESULTS: The five methods generated similar conclusions about the agreement between devices in the COPD example; however, some methods emphasized different aspects of the between-device comparison, and the interpretation was clearer for some methods compared to others. CONCLUSIONS: Five different methods used to assess agreement have been compared in the same setting to facilitate understanding and encourage the use of multiple agreement methods in practice. Although there are similarities between the methods, each method has its own strengths and weaknesses which are important for researchers to be aware of. We suggest that researchers consider using the coverage probability method alongside a graphical display of the raw data in method comparison studies. In the case of disagreement between devices, it is important to look beyond the overall summary agreement indices and consider the underlying causes. Summarising the data graphically and examining model parameters can both help with this.

The effect of temporal aggregation level in social network monitoring.

Social networks have become ubiquitous in modern society, which makes social network monitoring a research area of significant practical importance. Social network data consist of social interactions between pairs of individuals that are temporally aggregated over a certain interval of time, and the level of such temporal aggregation can have substantial impact on social network monitoring. There have been several studies on the effect of temporal aggregation in the process monitoring literature, but no studies on the effect of temporal aggregation in social network monitoring. We use the degree corrected stochastic block model (DCSBM) to simulate social networks and network anomalies and analyze these networks in the context of both count and binary network data. In conjunction with this model, we use the Priebe scan method as the monitoring method. We demonstrate that temporal aggregation at high levels leads to a considerable decrease in the ability to detect an anomaly within a specified time period. Moreover, converting social network communication data from counts to binary indicators can result in a significant loss of information, hindering detection performance. Aggregation at an appropriate level with count data, however, can amplify the anomalous signal generated by network anomalies and improve detection performance. Our results provide both insights on the practical effects of temporal aggregation and a framework for the study of other combinations of network models, surveillance methods, and types of anomalies.

Comparing heteroscedastic measurement systems with the probability of agreement.

Deciding whether two measurement systems agree well enough to be used interchangeably is important in medical and clinical contexts. Recently, the probability of agreement was proposed as an alternative to comparison techniques such as correlation, regression, and the limits of agreement approach, when the systems' measurement errors are homoscedastic. However, in medical and clinical contexts, it is common for measurement variability to increase proportionally with the magnitude of measurement. In this article, we extend the probability of agreement analysis to accommodate heteroscedastic measurement errors, demonstrating the versatility of this simple metric. We illustrate its use with two examples: one involving the comparison of blood pressure measurement devices, and the other involving the comparison of serum cholesterol assays.

Assessing agreement between two measurement systems: An alternative to the limits of agreement approach.

The comparison of two measurement systems is important in medical and other contexts. A common goal is to decide if a new measurement system agrees suitably with an existing one, and hence whether the two can be used interchangeably. Various methods for assessing interchangeability are available, the most popular being the limits of agreement approach due to Bland and Altman. In this article, we review the challenges of this technique and propose a model-based framework for comparing measurement systems that overcomes those challenges. The proposal is based on a simple metric, the probability of agreement, and a corresponding plot which can be used to summarize the agreement between two measurement systems. We also make recommendations for a study design that facilitates accurate and precise estimation of the probability of agreement.

Monitoring radiation use in cardiac fluoroscopy imaging procedures.

PURPOSE: Timely identification of systematic changes in radiation delivery of an imaging system can lead to a reduction in risk for the patients involved. However, existing quality assurance programs involving the routine testing of equipment performance using phantoms are limited in their ability to effectively carry out this task. To address this issue, the authors propose the implementation of an ongoing monitoring process that utilizes procedural data to identify unexpected large or small radiation exposures for individual patients, as well as to detect persistent changes in the radiation output of imaging platforms. METHODS: Data used in this study were obtained from records routinely collected during procedures performed in the cardiac catheterization imaging facility at St. Andrew's War Memorial Hospital, Brisbane, Australia, over the period January 2008-March 2010. A two stage monitoring process employing individual and exponentially weighted moving average (EWMA) control charts was developed and used to identify unexpectedly high or low radiation exposure levels for individual patients, as well as detect persistent changes in the radiation output delivered by the imaging systems. To increase sensitivity of the charts, we account for variation in dose area product (DAP) values due to other measured factors (patient weight, fluoroscopy time, and digital acquisition frame count) using multiple linear regression. Control charts are then constructed using the residual values from this linear regression. The proposed monitoring process was evaluated using simulation to model the performance of the process under known conditions. RESULTS: Retrospective application of this technique to actual clinical data identified a number of cases in which the DAP result could be considered unexpected. Most of these, upon review, were attributed to data entry errors. The charts monitoring the overall system radiation output trends demonstrated changes in equipment performance associated with relocation of the equipment to a new department. When tested under simulated conditions, the EWMA chart was capable of detecting a sustained 15% increase in average radiation output within 60 cases (<1 month of operation), while a 33% increase would be signaled within 20 cases. CONCLUSIONS: This technique offers a valuable enhancement to existing quality assurance programs in radiology that rely upon the testing of equipment radiation output at discrete time frames to ensure performance security.